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ABSTRACT 

The  project  goals  have  been  to  provide  enhanced  real-time  graphics  generation  capacity, 
computational  power,  and  real-time  audio  signal  processing  capability  for  the  Virtual  Environment 
Research,  Interactive  Technology,  And  Simulation  (VERITAS)  facility,  making  it  better  suited  to 
the  demands  of  DoD-relevant  research  projects  on  human  performance  in  complex  environments. 
VERITAS  is  owned  by  Wright  State  University,  but  housed  at  Wright-Patterson  AFB.  It  includes 
a  CAVE™,  which  is  an  immersive,  wide  field-of-view,  stereoscopic,  real-time  interactive  display 
system,  allowing  the  user  to  move  through  virtual  environments  with  minimal  encumbrances.  The 
CAVE™  is  controlled  by  a  Silicon  Graphics  Onyx™  computer  with  InfmiteReality ™  graphics. 

The  high-fidelity  simulations  in  this  facility  allow  a  variety  of  questions  related  to  human 
effectiveness  to  be  addressed.  The  DURIP  funds  were  used  to  purchase  three,  high-performance 
computer  subsystems:  a  multiprocessor  computational  subsystem,  a  graphics  generation 
subsystem,  and  an  acoustics  generation  subsystem.  These  subsystems  provide  critical  capabilities 
for  computationally  intensive,  real-time-constrained  applications,  including  simulation,  virtual 
environments,  auditory  and  visual  displays,  motor  control,  and  human  perception  and  cognition. 
This  instrumentation  has  supported  specific  funded  DoD  projects  investigating:  1)  display  and 
control  representations  for  UAV  operation,  and  2)  binaural  and  spatial  hearing. 
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OVERVIEW 

The  goals  of  this  project  have  been  to  provide  enhanced  real-time  graphics  generation 
capacity,  computational  power,  and  real-time  audio  signal  processing  capability  for  the  Virtual 
Environment  Research,  Interactive  Technology,  And  Simulation  (VERITAS)  facility.  VERITAS 
was  originally  constructed  using  funds  from  the  Ohio  Board  of  Regents  and  Wright  State 
University.  The  Air  Force  Research  Laboratory  (AFRL),  based  on  long-term  positive 
collaborations  with  Wright  State  University,  provided  space  to  house  the  facility.  Because  of 
budget  constraints  the  initial  hardware  and  software  configuration  for  the  facility  had  limited 
capability  and  was  somewhat  dated  when  installed.  The  DURIP  funds,  along  with  costsharing 
funds  from  Wright  State  University  and  the  Ohio  Board  of  Regents  Research  Challenge  Fund,  and 
additional  funds  from  AFOSR[F49620-95-l-0106],  were  used  to  purchase  three  state-of-the-art, 
high-performance  computer  subsystems:  a  multiprocessor  computational  subsystem,  a  graphics 
generation  subsystem,  and  an  acoustics  generation  subsystem.  Simultaneously,  additional  funds 
from  AFOSR  [F49620-97-1-0231],  along  with  costsharing  from  Wright  State  University  and  the 
Ohio  Board  of  Regents  Action  fund,  were  used  to  purchase  a  second  multiprocessor  computational 
subsystem,  haptic  interfaces,  a  control  PC  for  the  haptic  interfaces,  and  a  variety  of  software. 
Currently,  the  facility  is  well-suited  to  most  of  our  planned  work  on  display  and  control  designs  to 
reduce  cognitive  workload  in  UAV  operation  and  will  aid  our  work  on  auditory  displays.  We  plan 
to  continue  our  efforts  to  improve  the  VERITAS  Facility  to  assure  wide-ranging  capabilities  and 
state-of-the-art  resources. 

ENHANCEMENTS  TO  THE  VERITAS  FACILITY 

The  equipment  purchased  with  this  award  has  been  used  to  enhance  the  VERITAS  facility, 
which  is  owned  and  operated  by  Wright  State  University,  but  housed  at  the  Air  Force  Research 
Laboratory  at  Wright-Patterson  AFB.  This  facility  includes  a  CAVE rM,  which  is  a  highly 
immersive  wide  field-of-view,  stereoscopic,  real-time  interactive  graphical  display  system.  The 
CAVE ™  in  the  VERITAS  facility  consists  of  four  walls  and  a  floor,  each  10x10  feet,  all  of  which 
display  imagery  generated  by  a  Silicon  Graphics  (SGI)  Onyx™  multiprocessor  computer  with 
InfmiteReality™  graphics  and  projected  by  high-resolution  Electrohome  M8500  field-sequential 
stereoscopic  projectors  (rear-projection  for  the  four  walls,  top-projection  for  the  floor).  The  user’s 
head  and  hand  position  are  monitored  by  an  Ascension  Flock  of  Birds™  magnetic  tracker,  to 
provide  real-time  perspective  correction  (head  tracker)  and  real-time  interaction  with  virtual  objects 
(hand  tracker).  The  user’s  head  position  also  provides  the  basis  for  the  rendering  of  spatial  audio. 
Stereoscopic  images  can  be  seen  when  the  user  wears  Stereographies  Crystal  Eyes2™  LCD  shutter 
glasses. 

Hardware  Upgrades.  The  DURIP  funds  were  used  to  purchase  three  state-of-the-art, 
high-performance  computer  subsystems:  a  multiprocessor  computational  subsystem,  a  graphics 
generation  subsystem,  and  an  acoustics  generation  subsystem.  Additional  funds  were  used  to 
purchase  a  second  multiprocessor  computational  subsystem,  expansion  memory,  haptic  interfaces, 
workstations,  and  a  variety  of  software. 

A  multiprocessor  computational  subsystem  from  SGI  for  the  Onyx™  computer  at  the 
VERITAS  Facility  was  purchased  with  DURIP  funds  (a  second  subsystem  was  purchased  using 
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other  funds).  Each  subsystem  includes  four  64-bit  200-MHz  R10000  CPUs  (with  2  Mb  of 
secondary  cache  per  CPU).  These  eight  CPUs  replaced  the  four  32-bit  150-MHz  R4400  CPUs 
that  came  with  the  original  system.  The  R10000  represent  the  latest-generation  technology; 
whereas  the  four  R44000&  were  nearly  fully  utilized  for  graphics  control  alone,  with  only  limited 
resources  available  for  the  model  simulation  (e.g.,  flight  dynamics  models),  the  eight  RIOOOOs 
provide  a  significant  increase  in  the  computational  resources  available  for  simulation,  permitting  the 
implementation  and  evaluation  of  more  complex,  more  realistic  object  models,  high-fidelity 
simulation  of  flight  dynamics,  and  real-time  simulation  of  tactical  scenarios.  Also,  the  upgrade  to 
64-bit  CPUs  permits  us  to  fully  utilize  the  advanced  capabilities  of  the  64-bit  version  of  the  SGI 
IRIX  6.5  operating  system  that  is  required  for  full  support  of  the  InfiniteReality ™  graphics 
generation  subsystems. 

A  second  InfiniteReality™  graphics  generation  subsystem  from  SGI  was  added  to  the 
Onyx™  using  DURIP  funds.  Each  graphics  subsystem  has  an  output  video  bandwidth  of  624 
Mpixels  (textured,  multisampled,  antialiased,  and  z-buffered)  per  second,  can  generate  10.9 
million  polygons  per  second,  and  has  either  16Mb  (the  first  graphics  subsystem)  or  64Mb  (the 
second  graphics  subsystem)  of  texture  memory.  The  addition  of  the  second  subsystem  more  than 
doubles  the  graphics  capability  of  the  Onyx™.  The  InfiniteReality™  architecture  allows 
considerable  flexibility  when  determining  how  the  graphics  resources  are  distributed  across  the  five 
projectors  (four  walls  and  the  floor)  of  the  CAVE™.  The  trade-offs  among  total  number  of  active 
channels,  resolution,  refresh  rate,  etc.,  are  complex  but  are  supported  by  software  tools  on  the 
Onyx™  allowing  us  to  generate  a  wide  variety  of  video  combinations.  For  example,  the  CAVE™ 
floor,  four  walls,  and  a  workstation  monitor  for  the  user  can  all  be  simultaneously  driven  at  120- 
Hz  stereo  (60  Hz  per  eye)  with  a  resolution  of  800  X  800  pixels.  However,  higher-resolution  can 
be  achieved  by  reducing  the  number  of  displays  (e.g.,  four  simultaneous  1280xl024-pixel  120-Hz 
stereo  displays  on  four  surfaces  of  the  CAVE™ )  or  mixing  resolutions  (e.g.,  800x600  resolution 
on  rear  wall,  side  walls,  and  monitor,  coupled  with  1280x1024  resolution  on  the  front  wall  and 
floor). 

A  Huron-20  digital  acoustics  generation  subsystem  from  Lake  DSP  was  purchased  using 
DURIP  funds.  Before  the  purchase,  the  VERITAS  facility  had  no  dedicated  audio  generation 
capability.  The  Huron-20  is  configured  with  16  Motorola  56002  Digital  Signal  Processors,  each 
capable  of  20  million  (24  bit  X  24  bit)  multiply-and-adds  per  second.  It  can  apply  spatial  filtering 
to  128  virtual  sounds,  which  can  be  allocated  as  sources,  echoes,  or  reverberation.  It  can  be  used 
to  drive  headphones  or  up  to  16  loudspeakers  that  can  provide  unencumbered,  no-headphone, 
spatialized  audio  via  the  included  Lake  DSP  software.  The  Huron-20  and  Lake  DSP  software  is 
compatible  with  the  Paradigm  Vega  software  used  to  generate  simulations  in  the  CAVE™. 

Non-DURIP  funds  were  used  to  purchase  additional  main  memory  for  the  Onyx™.  The 
original  memory  configuration,  256MB,  was  adequate  for  the  four  R4400  CPU  configuration,  but 
the  upgrade  to  eight  R10000  CPUs  required  a  corresponding  increase  in  main  memory  to  512  MB. 

Non-DURIP  funds  have  been  used  to  purchase  input  devices,  including  a  haptic  interface. 
The  original  input  device  for  the  CAVE™  was  a  PinchGlove™,  which  permitted  the  capture  of 
only  simple  hand  gestures  (finger  closures  or  “pinches”).  A  passive  joystick  and  throttle  (F-22  Pro 
and  F16-TQS,  Thrustmaster ),  and  a  force-feedback  joystick  ( IE-2000,  Immersion),  which  is  being 
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used  to  provide  haptic  stimulation  (e.g,  force  cues  proportional  to  approach  and/or  glide  slope  error 
in  the  context  of  a  landing  task),  have  been  added.  Additional  funds  were  used  to  purchase  a 
Pentium  PC  for  control  of  the  IE-2000  haptic  joystick. 

Finally,  non-DURIP  funds  have  been  used  to  purchase  two  SGI  0  2  workstations  that 
permit  development  of  new  applications  for  the  CAVE™  while  experiments  are  running  on  the 
VERITAS  Onyx™. 

Software  Upgrades.  Non-DURIP  funds  have  been  used  to  purchase  upgrades  to  our 
high-performance  image  generation  software,  software  for  high-fidelity  simulation  of  flight 
dynamics,  and  software  for  generation  and  simulation  of  tactical  engagement  scenarios. 

The  software  used  to  generate  the  visual  simulations  in  the  CAVE ™  is  Paradigm 
Simulation,  Inc.  ’s  (PSI)  Vega.  Initially,  VERITAS  had  a  single  base  license  for  Vega.  Additional 
base  Vega  licenses  for  program  development  and  Vega  modules  needed  to  support  various  research 
projects  have  been  added.  Specifically,  the  VCR  module  is  used  to  collect  data  by  recording 
simulation  information  on  a  frame-by-frame  basis;  the  Symbology  module  allows  simulation  data 
to  drive  virtual  instruments  (e.g.,  primary  flight  display);  the  Large-Area  Database  Management 
module  allows  simulations  to  use  terrain  databases  (“virtual  worlds”)  that  are  larger  than  main 
memory;  AudioWorks  interfaces  the  simulation  to  the  Huron-20  digital  audio  subsystem. 
Additional  Vega  base  licenses  were  purchased  to  permit  the  simultaneous  operation  of  the  CAVE™ 
and  applications  development  on  the  two  SGI  O  2  workstations.  We  also  obtained  from  PSI  a  geo¬ 
typical  terrain  database  {Vampire),  60km  x  80km  in  extent,  providing  a  wide  variety  of  terrain 
types,  suitable  for  out-the-window  scene  generation  in  the  simulation  of  high-velocity,  low-altitude 
flight. 

For  research  projects  requiring  high-fidelity  simulation  of  flight  dynamics,  we  obtained 
Virtual  Prototypes,  Inc’s  (VPI)  FLSIM,  which  provides  both  a  dynamics  simulation  engine  and  a 
database  editor  that  is  used  to  graphically  display  and  modify  the  specific  aerodynamics  data  of  the 
vehicle  to  be  simulated.  Aircraft  databases  provided  with  FLSIM  include:  F-16,  KC-135,  B-747, 
and  C-140.  The  graphical  database  editor  simplifies  the  creation  of  aerodynamics  databases  for 
other  fixed-wing  vehicles. 

Also  from  VPI,  we  obtained  the  tactical  scenario  generator  and  database  editor  STAGE, 
which  is  used  to  simulate  the  behaviors  of  targets  and  threats  in,  for  example,  a  simulated 
integrated  air  defense  system.  STAGE  provides  the  ability  to  import  the  terrain  database  used  for 
visual  simulation  in  Vega  and  incorporate  the  resulting  geophysical  constraints  in  the  tactical 
simulation.  The  graphical  database  editor  simplifies  the  creation  of  scenarios  containing  specific 
instances  of  a  variety  of  targets  and  threats. 

Both  FLSIM  and  STAGE  are  designed  to  be  extensible  and  integrated  into  larger 
simulations.  We  obtained  from  VPI,  user-module  code  that  provides  this  integration  with  our 
Vega  visual  simulation.  The  CPU  upgrade  allows  for  one  CPU  to  run  the  FLSIM  flight  dynamics 
simulation,  and  another  CPU  to  run  the  STAGE  tactical  scenario  simulation,  with  the  remainder 
dedicated  to  the  Vega  visual  simulation. 

Current  Status.  These  upgrades  have  been  selected,  evaluated,  purchased,  installed, 
and  integrated.  The  multiprocessor  computational  subsystems,  along  with  the  additional  memory, 
allows  more  complex  simulations  to  run  at  higher  frame-rates,  including  realistic  flight-dynamics. 
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The  additional  graphics  generation  subsystem  more  than  doubles  the  video  resolution  and  more 
fully  utilizes  the  capacity  of  the  projectors.  The  acoustics  generation  subsystem  will  allow  us  to 
present  complex  auditory  environments,  including  acoustic  reflections  and  reverberation.  The 
haptic  stick  allows  us  to  examine  various  strategies  using  force-feedback  to  provide  information  to 
operators  (e.g.,  “force  tunnels”).  The  two  additional  workstations  allow  simultaneous  program 
development  and  experimentation.  New  software  provides  high-level  tools  and  off  the  shelf 
solutions  to  a  number  of  identified  project  needs  and  have  allowed  us  to  implement  a  high-fidelity 
simulation  with  realistic  (as  indicated,  for  example,  by  the  reports  of  pilots)  flight  dynamics. 

VERITAS  as  currently  configured  is  one  of  the  top  projected-virtual  environment  facilities 
in  the  world.  It  offers  wide-ranging  capabilities,  but  has  been  designed  so  that  programmers  with 
relatively  little  experience  in  computer  graphics  and  simulation  can  still  create  and/or  modify 
complex  virtual  environments,  flight  simulations,  and  tactical  scenarios.  Although  every  virtual 
environment  generation  strategy  has  limitations,  VERITAS  is  well-suited  to  our  DoD-related 
research,  allowing  the  integration  of  real  (e.g.,  the  user’s  hand  or  a  cockpit  mock-up)  and  virtual 
objects,  3D  rendering  of  visual  displays  such  that  both  near  (e.g.,  cockpit  instruments)  and  distant 
(terrain)  virtual  objects  can  be  simultaneously  appreciated,  realistic  spatialized  audio,  and  haptic 
feedback  via  stick  forces.  The  facility  can  be  configured  to  provide  a  compelling  pilot-centered 
flight  simulation  with  realistic  flight  dynamics,  but  can  also  be  used  to  simulate  other  command  and 
control  representations,  including  traditional  operator  workstation,  “data  wall,”  and  3D  God’s- 
eye/sand-table  for  tactical  and  strategic  planning. 

SUPPORTED  PROJECTS 

Much  of  the  current  work  at  the  VERITAS  facility  is  funded  by  a  grant  from  the  Air  Force 
Office  of  Scientific  Research  [F49620-97- 1-0231]  and  focuses  on  the  design  and  evaluation  of 
displays  and  controls  for  the  operation  of  Uninhabited  Aerial  Vehicles  (UAV).  This  project  will 
establish  necessary  and  sufficient  information  constraints  for  effective  man-in-the-loop  control  of 
UAVs,  and  thereby  help  to  specify  the  range  of  missions  and  operational  environments  where 
UAVs  can  be  reliably  used.  This  collaborative  project  involves  several  faculty  members  from  the 
Department  of  Psychology  (Kevin  Bennett,  John  Flach,  Robert  Gilkey,  Scott  Isabelle,  Valerie 
Shalin)  and  their  graduate  students  (Jeff  Calcaterra,  Michael  Payne,  Matthew  Smith,  Billy  Jack 
Crawford,  Brian  Simpson,  Paul  Jacques),  and  two  faculty  from  the  College  of  Engineering  (Jer- 
Sen  Chen,  Jennie  Gallimore)  and  their  graduate  students  (Jessie  Lucas,  Nathan  Brannon,  Arti 
Kulkarni,  Lakshmi  Yechuri,  Xing  Xu).  Interactions  with  Air  Force  personnel  have  been 
encouraged  by  this  project  (Sam  Schiflet  and  Linda  Elliot  at  Brooks  AFB,  Elizabeth  Martin  at 
Williams  AFB,  Daniel  Repperger  and  Michael  Haas  at  WPAFB,  Curtis  Spenny  at  AFIT).  Haas, 
Repperger,  and  Spenny  are  currently  collaborating  with  members  of  our  group  on  related  projects. 
Interactions  with  Navy  personnel  (LtCmdr  Fred  Patterson  at  NAMRL  Pennsicola)have  also  been 
enhanced  by  this  project. 

This  project  strives  to  develop  a  basic  understanding  of  cognitive  workload  and  situation 
awareness  relative  to  the  performance  and  functionality  of  complex  socio-technical  systems. 

Toward  this  end  we  are  implementing  a  complex  synthetic  task  environment  that  includes  critical 
elements  of  a  SEAD  mission;  we  are  performing  and  evaluating  a  multi-component  task  analyses  of 
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the  SEAD  mission  and  various  telerobotic  systems,  in  order  to  identity  critical  task  components 
and  sources  of  cognitive  workload;  we  are  performing  targeted  studies  of  sensory  and  perceptual 
capabilities  and  limitations  that  may  impact  cognitive  workload;  we  are  designing  display  and 
control  interfaces  that  directly  represent  the  functional  impact  of  commands  and  the  semantics  of 
environmental  information,  and  will  evaluate  these  displays  in  terms  of  their  influence  on 
performance  and  cognitive  workload;  and  we  will  evaluate  the  constraints  on  human  performance 
and  cognitive  workload  in  telerobotic  and  virtual  display  systems  such  as  Uninhabited  Aerial 
Vehicles  (UAVs),  including  level  of  automation,  bandwidth,  delays,  and  reliability. 

Research  on  spatial  hearing  funded  by  AFOSR[F49620-95-l-0106]  has  also  benefitted 
from  the  equipment  purchased  for  VERITAS.  This  project  is  using  psychophysical  measurement 
and  modeling  to  determine  the  acoustic  cues  that  mediate  human  sound  localization  in  real  and 
virtual  environments.  The  project  has  supported  Robert  Gilkey  and  Scott  Isabelle,  and  graduate 
students  Brian  Simpson  and  James  Kondash.  The  Huron-20  has  been  used  to  analyze  room 
acoustics,  such  that  realistic  echoes  and  reverberation  can  be  presented  in  virtual  environments  at 
VERITAS.  Although  funding  for  this  project  has  now  ended,  our  efforts  to  determine  the  impact 
of  room  acoustics  on  localization  performance,  multichannel  communication,  and  the  sense  of 
presence  in  virtual  environments  will  continue. 

A  number  of  other  projects  are  also  benefitting  from  the  enhancements  to  VERITAS. 
These  projects  examine  issues  such  as  the  use  of  virtual  environment  technology  in  medicine,  data 
visualization,  and  collision  avoidance. 
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